Long distance and metropolitan area network (MAN) communications rely on short-haul and long haul fiber optic networks to transport data and telephony traffic. One conventional way to transmit data in fiber networks is through a Synchronous Optical Network/Synchronous Digital Hierarchy (SONET/SDH) protocol. In a SONET/SDH network, data travels in fixed size envelopes that repeat every 125 microseconds. With this synchronous fixed-length framing, every byte (e.g., 8 bits of data) inside a SONET/SDH frame represents a 64 Kbps (64000 bits/sec) channel. The 64 Kbps channel has the same rate as supported by current telephone channels (also called DS0 channels).
SONET was designed to efficiently carry telephony Plesiochronous Digital Hierarchy (PDH) channels such as T1/T3. This was easily achieved by dividing the payload area in fixed slots called virtual tributaries (VT). These virtual tributaries are then grouped together to form higher-rate channels. These fixed slots are efficient for carrying fixed-bandwidth telephony channels because any one or more channels can be added or removed from a bundle without processing an entire payload of channels. Because SONET frames repeat at fixed intervals, these virtual tributaries have fixed locations and time intervals, and it is easy to extract T1/T3 or fractions of T1 without processing the entire SONET payload.
With growing volume in data traffic, however, SONET/SDH networks must now carry a significantly large number of data packets—such as ATM (Asynchronous Transfer Mode—53 bytes each) and IP (Internet Protocol—variable-size packets) in addition to traditional T1/T3 channels. The synchronous framing structure of SONET/SDH that is quite efficient for carrying T1/T3 channels is not able to carry both fixed-bandwidth and variable-bandwidth channels in an optimum way.
SONET/SDH has an inefficient utilization of fiber bandwidth for data packets. For data transport, some of the virtual tributaries are created for transporting fixed-bandwidth T1 traffic while others are used for transporting packet data packets such as ATM and IP. Since an individual virtual tributary has a limited bandwidth, extra mechanisms have to be used for sending data packets of higher bandwidth using virtual tributaries.
In one technique, a 10 Mbps data packet channel, for example, is inverse-multiplexed into smaller bandwidth streams and then sent on many virtual tributaries. At the other end, these streams are integrated to reconstruct the full 10 Mbps channel. In another method, many of the virtual tributaries are concatenated using hardware to create a higher-bandwidth virtual tributary for transmitting the high-bandwidth data packet.
SONET/SDH lacks of support for data mixing. A SONET fiber link carrying frames containing ATM cells cannot carry POS, because ATM cells frequently carry QoS-sensitive data such as CES (Circuit Emulation Service) or multimedia traffic. Introduction of SONET frames containing POS will cause significant delays (e.g., 125 μS for each POS frame inserted in the link).
In each of these methods, a unique Path Signal Label (PSL) value in the POH (Path Over-Head) field of the SONET frame identifies the type of data transmission inside the payload. The payload area is also referred to as SPE (Synchronous Payload Envelope). Because a PSL value identifies contents of the entire SONET payload envelope, only one type of transmission can be supported at a time in a SONET frame.
One method for data transmission is to use the entire SONET SPE for data packets. The SONET payload area is filled with IP packets using Packet-over-SONET (POS) packets. POS packets are packets deliminated by 0×7E (Hexadecimal) at both ends of a packet, with a framing using PPP (Point-to-Point Protocol). Many packets can be put inside a single SONET SPE. This method can only support variable-length packet protocol such as IP. A SONET fiber containing these packets cannot transport T1/T3 channels or real-time streams using ATM cells. The reason for this limitation is that each SONET SPE containing IP packets, for example, introduces a delay of 125 microseconds. Such a delay is not acceptable for T1/T3 circuits or real-time streams using ATM cells.
Another method for data transmission is to use the entire SONET SPE for ATM cells. In this case, a SONET SPE is filled with ATM Cells. ATM cells are delimited by their fixed length, and are tracked by doing a hunt for their header checksum byte. Services such as T1, Frame Relay, Ethernet, etc. are transported over ATM using standard protocols. This requires complex implementations in hardware and incorporation of ATM service interworking at each service boundary.
Another method for data transmission is to use the virtual tributaries (VT) for data packets and ATM cells. In this method, a SONET SPE is partitioned in many fixed-bandwidth slots called virtual tributaries (VT). For data transport, some of these virtual tributaries may contain T1/T3 type of fixed-bandwidth traffic while others are used for transporting packet data packets such as ATM and IP.
Since an individual virtual tributary has a limited bandwidth, extra mechanisms have to be used for sending data packets of higher bandwidth using virtual tributaries. In one technique, a 10 Mbps data packet channel, for example, is inverse-multiplexed into smaller bandwidth streams and then sent on many virtual tributaries. At the other end, these streams are integrated to reconstruct the full 10 Mbps channel. In another method, many of the virtual tributaries are concatenated using hardware to create a higher-bandwidth virtual tributary for transmitting the high-bandwidth data packet.
Each of these methods uses a fixed-bandwidth channel or a set of channels for transmitting network data packets. In each method, bandwidth capacity of the fiber is poorly utilized since network data packets are bursty in nature and average bandwidth utilization is quite low.
Referring to FIG. 1, examples of various data types are shown. A set of time-division-multiplexed (TDM) packets 12a–12n, a set of ATM packets 14a–14n and a set of POS packets 16a–16n are shown in connection with a SONET fiber line 18. In a SONET network, only one type of data can be transferred at a time. The data is identified by a unique PSL (path signal label) byte value inside a Path Over-Head (POH) of the TDM packets 12a–12n, the ATM packets 14a–14n, the POS packets 16a–16n, of PDH traffic. The nodes at different points in the SONET fiber line 18 have different types of data to send on the network.
Referring to FIG. 2, an example of a system 20 illustrating problems in transmitting higher bandwidth data packets with fixed bandwidth slots is shown. Transmission of fixed and variable-bandwidth packets in SONET/SDH frames is shown. SONET/SDH was developed for efficiently transporting telephony signals over long links. In order to support timing of the smallest telephony components DS0 (e.g., 64 Kbps), SONET/SDH frames are generated using fixed length packets repeated at intervals of 125 μS.
At such a transfer rate, each byte of a SONET/SDH frame represents a basic telephony channel, often represented as DS0. A number of such bytes are reserved to form higher-order PDH signals (such as T1) which use 28 DS0 channels. One such method for supporting telephony channels is by dividing the payload into a number of virtual tributaries (VT).
To achieve precise timing, PDH bytes must begin at the same offset inside a SONET SPE 22a–22n. If data packets are transmitted along with the PDH channels, the data must be sent in the fixed length slots. Allocation of PDH channels at different locations inside the SONET SPE 22a–22n creates fragments of unused bytes all over the SPE. If the slots B/D/F/G/H/I are used for variable-length data packets, the slots cannot be fully used because the fixed bandwidth slots, such as C and E, do not allow the packets to continue without fragmenting while crossiing slot boundaries.
Two conventional approaches have been used to accommodate high bandwidth signals along with low bandwidth signals. A first conventional approach is virtual concatenation. A second conventional approach is inverse multiplexing.
Referring to FIG. 3, an example of a conventional virtual concatenation approach is shown. To transport fixed bandwidth traffic (e.g., T1/T3), the SONET SPE payload area is divided into fixed timeslots (i.e., virtual tributaries). An example of a virtual tributary is the standard VT1.5. A VT1.5 can carry 1.5 Mbps of information for each tributary, which is equal to a T1 channel.
Allocation of bandwidth for LAN data (such as 10/100 Mbps Ethernet) using a SONET fiber network and VT becomes difficult. For example, one has to either dedicate an entire STS-1 (51 Mbps), or use several VT channels with inverse multiplexing (to be described in more detail in connection with FIG. 3).
Virtual concatenation combines several VT channels into bigger virtual pipes to carry higher bandwidth traffic. With such a protocol, some virtual tributaries can carry T1/T3 data as usual while others are concatenated for transport of higher bandwidth data traffic such as 10/100 Mbps. When many VT channels are concatenated, a fatter pipe of a higher bandwidth may be generated that can carry the entire bandwidth of a LAN. As a result, splitting a higher bandwidth into smaller VT channels and having to regroup them to get the LAN traffic can be avoided. However, virtual concatenation has one or more of the following disadvantages:                (i) virtual concatenation allocates a fixed bandwidth for LAN, but cannot dynamically adjust bandwidth usage on a packet-by-packet basis;        (ii) bursty LAN traffic bandwidth usage is typically quite low, which can result in significant waste when used over a fixed bandwidth pipe (e.g., the average use of a 10 Mbps link is 20% and, if an entire STS-1 is used, the efficiency becomes about 4%); and/or        (iii) while some virtual pipe may be overloaded with traffic others may be underused (i.e., virtual concatenation cannot dynamically adjust network loads on different channels).        
Referring to FIG. 4, an example of the inverse multiplexing approach for transmitting higher bandwidth channels is shown. Inverse multiplexing is similar to the virtual concatenation. However, different SONET virtual tributaries (e.g., LINK#1–LINK#N) are not concatenated but are used as separate conduits of data transfer.
Data from a high bandwidth pipe, such as a 10 Mbps Ethernet LAN, can be sent over the multiple VT1.5 tributaries LINK#1–LINK#N using inverse multiplexing. The data is later recovered using standard inverse multiplexing protocols. Inverse multiplexing suffers from the same problems of bandwidth maximization as the virtual concatenation approach. In particular, bandwidth needs to be reserved in advance and all of the available bandwidth is not usable.
In particular, inverse multiplexing suffers from one or more of the following disadvantages:                (i) inverse multiplexing allocates a fixed bandwidth for LAN, but cannot dynamically adjust bandwidth usage on a packet-by-packet basis; and/or        (ii) bursty LAN traffic bandwidth usage is typically quite low, resulting in a significant waste when used over a fixed bandwidth pipe (e.g., if the average use of a 10 Mbps link is 20% and if an entire STS-1 is used, then the efficiency becomes about 4%).        
Similar to virtual concatenation, while one VT (e.g., LINK#1) may be overloaded with traffic, other VTs (e.g., LINK#2–LINK#N) may be underutilized. Therefore, the bandwidth cannot be dynamically adjusted on different VT channels.
Referring to FIG. 5, an example of ATM VP multiplexing is shown. ATM VP multiplexing provides another conventional approach to attempt to fully utilize SONET bandwidth. ATM VP multiplexing fills the payload area with ATM cells, where packets are encoded in ATM cells and then placed inside a SONET SPE.
ATM Circuit Emulation Service (CES) is normally used for carrying PDH traffic such as T1. Data packets are transmitted using Multiprotocol-over-ATM encapsulation methods.
ATM VP multiplexing suffers from one or more of the following disadvantages:                (i) operations, administration and maintenance (OAM) operations of an ATM network are different from that of a SONET network, and management of the two networks can become difficult;        (ii) ATM network routing and switch-to-switch signaling of data paths are different from IP network routes, resulting in network operational complexity; and/or        (iii) a high percentage of network traffic consists of IP packets with small packet sizes of around 40 bytes. With IP over ATM (e.g., rfc2684—Multiprotocol Encapsulation over ATM AAL5), payload size slightly exceeds what can fit inside a single ATM cell. Such an excess results in transmission of two ATM cells, with the second cell that is mostly ATM overhead and stuffing bytes. The transmission of two ATM cells, along with other ATM overhead, requires allocating extra SONET bandwidth for IP transport.        
Referring to FIG. 6, various data transmissions are shown in a SONET ring 30. Assume, for example, that a node A has ATM cells 14a–14n to transmit on the SONET ring 30. The SONET ring 30 forms a SONET synchronous payload envelope (SPE), sets the PSL value for ATM cells (in the POH) and sends the SONET frame down a link 32a–32n. Even if the SPE is only partially full, the entire SPE frame size is transmitted. If a node B has IP packets to send, the node cannot use the partially filled SONET frame received from the node A to add POS packets, because the PSL value identifies only one type of data (ATM cells, in this example). The node must now wait 125 μS for the partially filled packet to transfer down the link 32b–32n. Furthermore, the SONET ring 30 cannot handle T1 or T3 channels entirely in a payload space. The only way it can simultaneously support T1/T3 channels is by using virtual tributaries and by using some of the tributaries for non-T1/T3 traffic.
Statistically, a significant percentage of network traffic comprises IP packets that are quite small in size. Because higher-speed concatenated SONET frames are large in size, many times a SONET SPE may not be completely filled with packets or cells at the origin. Because a SONET SPE carries only one type of data, the originating SONET node or other nodes downstream will not be able to add any different type of data to the SPE, and the SONET frame may be heavily underutilized. Such an under-utilization of the SONET frame degrades the overall bandwidth utilization of fiber capacity with SONET/SDH.
Referring to FIG. 7, a system 50 is shown implementing multiple fiber optic networks. The system 50 is shown implementing a T1 network 52, an ATM network 54 and a POS network 56. Referring to FIG. 8, a timing diagram illustrating transfer times of the multiple data types of FIG. 7 is shown. Separate fiber networks for data types with current SONET protocols and separate fiber links are needed for transporting ATM, PDH traffic and variable-length packets such as IP or POS. PDH channels are created by strict timing relationships between consecutive Synchronous Transport Signal, Level 1 (STS-1) frames (to form a superframe). Any intervening STS-1 frames containing ATM or POS packets will violate such timing specifications.
Various conventional protocols have been developed to improve bandwidth usage by attempting to partially solve two problems in SONET networking (i) lack of support for data mixing, and (ii) bandwidth reuse limitations. Such conventional protocols have been partially able to achieve additional bandwidth either by creating fatter pipes with VT, by filling a payload with ATM cells carrying different types of data, or by using a SONET link as a multi-node access network using Ethernet framing for data packets.
While virtual tributaries provide an efficient way to transport PDH traffic, allocation of bandwidth for LAN data such as 10/100 Mbps using a SONET fiber containing T1 traffic becomes difficult. To support 10 Mbps through Virtual tributaries one has to either dedicate an entire STS-1 (51 Mbps) frame, or use several virtual tributary channels and to perform inverse multiplexing.
Virtual concatenation concatenates several VT channels into bigger virtual pipes to carry higher bandwidth traffic. With such a protocol, while some virtual tributaries carry T1/T3 data as usual, others are concatenated for transport of higher bandwidth data traffic such as 10/100 Mbps links.
However, virtual concatenation allocates a fixed bandwidth for LAN. Virtual tributaries cannot dynamically adjust bandwidth usage on a packet-by-packet basis. While it is possible to change the concatenated bandwidth through software, such an implementation does not yield much for bandwidth utilization. Bursty LAN traffic bandwidth usage is typically quite low, resulting in significant waste when used over a fixed bandwidth virtual pipe (average use of a 10 Mbps link is 20%, and if an entire STS-1 is used the efficiency becomes about 4%). Another problem with virtual concatenation is that while one virtual pipe may be overloaded with traffic, others may be underutilized. Virtual concatenation cannot dynamically adjust network loads on different channels.
Another conventional approach to increase the utilization of SONET bandwidth is to fill the payload area with ATM cells using a technique known as ATM VP (Virtual Path) multiplexing. ATM VP multiplexing encodes packets in ATM cells and then inserts the cells inside a SONET SPE. The ATM VP Multiplexing typically utilizes CES to carry DS0/1 PDH traffic.
However, operations, administration and maintenance (OAM) operations of ATM networks are different from that of SONET, and management of the two protocols can become difficult. Similarly, ATM network routing and switch-to-switch signaling data paths are different from IP network routes, resulting in network operational complexity.
Network traffic statistics monitoring has shown that a significant percentage (about 45%) of network traffic comprises IP packets with small packet sizes (e.g., around 40 bytes). With IP over ATM (rfc2684—Multiprotocol Encapsulation over ATM AAL5) payload size slightly exceeds what can fit inside a single ATM cell. Such an arrangement results in transmission of two ATM cells, with the second cell that is mostly ATM overhead and stuffing bytes. This, with other ATM overheads, means having to allocate extra SONET bandwidth for IP transport if ATM is used as a transport protocol.
In addition, using ATM for sending different services requires implementation of ATM transport protocols and interworking for all related protocols (such as IP-over-ATM, Frame Relay-ATM Internetworking, circuit emulation, etc.) in the device. Such implementation requires a high level of complexity in hardware and software, resulting in higher costs of manufacturing and operation of networks.
Such conventional network architectures are not efficient for transmitting variable-length IP packets that form the majority of data network traffic on the Internet. Data network traffic comprises variable-length IP packets and fixed-length ATM cells (i.e., 53 bytes).
As more and more data is being transported on SONET/SDH rings, there is a need to send variable-length packets on pre-existing SONET/SDH networks. These packets originate out of routers and other data access devices. While SONET/SDH networks must transport these data packets, they must also continue to support TDM-style fixed length packets for telephony and leased line applications.
Conventional approaches cannot mix POS with ATM cells in a single SONET SPE. Conventional approaches do not leave some area of the SONET SPE reserved for VTs and others for POS and/or ATM cells.
Limitations of conventional approaches include one or more of the following (i) inability to mix TDM channels with packet-oriented data over SONET/SDH rings due to timing constraints of the TDM channels without a fixed bandwidth virtual tributary mechanism and (ii) limiting data channels sent on fiber carrying T1 lines to Virtual tributaries (since virtual tributaries are of fixed bandwidth, this restriction limits data channels to fixed bandwidth operation).
Therefore, if a SONET SPE is carrying all ATM cells, it cannot carry IP variable-length packets, and vice versa. Such switches route all packets coming at one tributary to another until switching paths are changed through reprogramming.
Conventional approaches do not take into account network loading conditions and do not support dynamic bandwidth provisioning. Conventional approaches also have increased traffic on core links from too much concentration into too few links. Transporting IP traffic over virtual tributary channels that was originally designed for DS0/1/3 connectivity is inefficient. Because VT assignment is fixed, IP transport is not able to take advantage of total available SONET/SDH bandwidth.
In conventional approaches, IP packets are constrained to go through some pre-configured VT channels while other VT channels may be under-utilized. Once a VT channel is dedicated for a particular traffic and is put on a specific circuit-switched path, the topology does not change, even if traffic conditions change.
Conventional approaches have the following disadvantages (i) ATM and Packets are implemented on different rings because of QoS and timing issues; (ii) very high cost for new fiber and SONET equipment for Telco/ISP/MAN; (iii) only one type of packet goes inside a SONET SPE at one time (the remaining bytes of the frame with conventional SONET are wasted) (SONET packets go around the entire SONET ring, limiting bandwidth); and/or (iv) the only way to support telephony channels along with data packets is to allocated part of SONET frame for packet data transmissions (which results in an inefficient bandwidth usage).